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I.  Brief  Review  of  Concepts  in  Cultural  Consensus  Theory 

Cultural  Consensus  Theory  is  a  statistical  modeling  approach  to  information  pooling 
developed  by  William  H.  Batchelder,  A.  Kimball  Romney,  Susan  Weller,  and  colleagues. 
It  first  appeared  in  the  mid  1980s  in  a  series  of  papers  (Batchelder  and  Romney,  1986, 
1988,  1989;  Romney,  Weller,  and  Batchelder,  1986;  Romney,  Batchelder,  and  Weller, 
1987).  Since  its  inception,  CCT  has  become  a  leading  methodology  in  cultural 
anthropology,  e.g.  Romney  and  Batchelder  (1999),  Weller  (2007),  and  in  the  1990s  and 
early  2000s  a  few  additional  papers  on  CCT  were  published,  some  applying  it  in  other 
areas  of  the  social  sciences.  However,  in  the  last  five  years,  there  have  been  major 
developments  in  CCT  models  supported  by  grants  from  the  AFOSR  and  the  ARO.  In 
particular,  during  the  period  of  the  three-year  grant  from  the  ARO,  CCT  has  seen 
applications  in  quantitative  journals  in  social  networks,  sociology,  psychology,  education, 
and  statistics  (described  later). 

CCT  consists  of  a  collection  of  cognitively  motivated,  parametric  statistical  models  for 
pooling  the  responses  of  “informants”  (experts,  respondents)  to  a  series  of  questions  about 
some  domain  of  their  shared  knowledge  or  beliefs.  CCT  domains  include  areas  such  as 
eyewitness  reports,  forecasts  of  probabilistic  events,  folk  medical  beliefs,  grammaticality 
judgments,  moral  or  religious  beliefs,  scientific  knowledge  in  various  populations,  judges 
grading  student  essays  or  sports  performance,  and  ties  in  a  social  or  covert  network.  CCT 
is  intended  to  operate  with  relatively  small  groups  of  heterogeneous  informants,  and  it 
applies  in  situations  where  researchers  know  how  to  pose  questions  in  some  response 
format  to  the  informants,  but  they  do  not  know  the  ‘consensus  answers,’  if  any,  a  priori. 
The  data  structure  of  CCT  requires  that  each  informant  answer  each  of  a  fixed  series  of 
questions  (procedures  have  been  developed  for  dealing  with  missing  data  as  well). 

The  overall  aims  of  a  CCT  model  can  be  described  in  general  terms  as  follows.  Let  X-^hQ 
a  random  variable  representing  the  response  of  informant  i  to  question  k,  and 
X  =  the  random  response  profile  matrix  for  N  informants  each  answering  M 
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questions.  The  goal  of  applying  a  CCT  model  to  such  data  includes  the  following:  (a) 
Identify  one  or  more  latent  “cultural  groups”  of  informant  that  share  consensus  answers  to 
the  questions;  (b)  Decide  if  the  statistical  model  used  to  do  this  is  sufficiently  supported  by 
the  data,  (c)  If  statistical  support  for  the  model  is  found,  then  the  goal  is  to  estimate  the 
parameters  of  the  model.  Most  CCT  models  have  parameters  for  the  consensus  answers  to 
the  questions,  the  “cultural  salience”  (difficulty)  of  each  question,  the  “cultural 
competence”  (calibration)  of  each  informant,  the  “response  biases”  of  each  informant.  If 
more  than  one  consensus  truth  is  estimated,  corresponding  group  membership  parameters 
for  each  informant  are  estimated.  It  is  important  to  note  that  all  CCT  inference  is  based 
only  on  the  obtained  data  X,  and  no  exogenous  information  is  used  in  the  estimation. 

At  the  time  of  the  proposal,  the  only  generally  available  statistical  software  for  a  CCT 
model  was  due  to  Stephen  P.  Borgatti  (1900,  1992).  This  software  was  for  an  early  CCT 
model  called  the  General  Condorcet  Model  (GCM,  Batchelder  and  Romney,  1988)  for 
dichotomous  (True/False)  questionnaire  data  (it  could  also  be  applied  to  multiple  choice 
data  with  additional  assumptions).  Also  the  model  assumed  only  one  consensus  answer 
key,  and  it  assumed  that  all  items  were  equally  difficult  and  that  all  informants  had  neutral 
(50:50)  guessing  biases.  While  many  published  articles  in  cultural  anthropology  made  use 
of  this  software  (e.g.  Weller,  2007),  it  was  obvious  that  there  was  a  need  for  models  to  be 
developed  for  other  testing  formats,  e.g.  ordinal  (Likert)  data,  continuous  data;  and  in  each 
case  it  was  important  to  have  available  user-friendly  software  for  the  new  models.  Some 
effort  in  this  direction  was  provided  for  the  GCM  in  Karabatsos  and  Batchelder  (2003), 
but  the  inference  software  was  not  presented  in  a  way  that  unsophisticated  users  could 
employ  it,  and  as  a  consequence  it  had  almost  no  effect  on  the  practice  of  users  of  CCT. 
These  limitations  were  the  main  basis  of  what  motivated  this  grant  that  was  funded  by  the 
Army  Research  Office. 

II.  New  Statistical  Software 

The  original  proposal  stated  the  following:  “Despite  its  popularity,  CCT  is  handicapped  by 
the  availability  of  very  limited  software  that  can  handle  only  special  cases  of  the  theory. 
The  principle  objective  of  the  proposal  is  to  develop  a  user-friendly  and  freely  available 
software  package  for  cultural  consensus  theory  (CCT)  based  on  modern  computational 
Bayesian  approaches  to  statistical  inference.  “ 

There  have  been  three  major  developments  supported  by  the  ARO  grant  in  this  area.  First, 
in  two  papers  to  appear  in  Field  Methods  (Papers  10,  11),  the  weaknesses  in  the  previous 
software  for  the  GCM  have  been  overcome.  Paper  (10)  provided  the  Bayesian  theory  and 
the  basics  of  a  Graphic  Users  Interface  (GUI)  for  the  GCM,  and  paper  (11)  was  a  User’s 
Guide  for  the  GUI.  Field  Methods,  a  leading  journal  for  cultural  anthropology  studies, 
accepted  this  paper  in  2012;  however,  the  large  backlog  in  that  journal  prevented  its 
publication  until  now  (available  now  On-line,  Field  Methods,  February  10,  2014). 
However,  the  two  papers  have  been  available  on  Dr.  Zita  Oracevz’s  website 
(bayesian.zitaoravecz.net)  since  2012,  and  she  offers  help  to  people  trying  to  use  the  GUI 
(Zita  is  the  Post  Doctoral  Fellow  supported  by  the  grant).  The  style  of  the  paper  and  the 
GUI  is  to  make  it  understandable  and  usable  by  researchers  with  limited  statistical  and 
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computational  backgrounds.  This  software  improves  upon  the  Borgatti  (1990,  1992) 
software  in  four  important  respeets:  (1)  It  uses  state  of  the  art  Bayesian  statistieal 
inferenee,  the  eurrent  standard  in  computational  statistics,  (2)  It  allows  items  to  differ  in 
cultural  saliency,  (3)  It  allows  informants  to  differ  in  guessing  biases,  (4)  It  provides  a 
statistieal  sound  Bayesian  post  predietive  test  for  the  single  eulture  assumption  of  the 
model.  Number  (4)  is  important  because  it  eorrects  a  flaw  in  the  earlier  software,  where 
the  one-eulture  assumption  was  not  assessed  properly. 

A  seeond  place  where  new  Bayesian  software  was  provided  during  the  period  of  the  ARO 
grant  was  that  the  early  GCM  and  later  models  for  other  response  formats  (to  be  deseribed 
in  Seetion  III)  were  formulated  hierarchically,  and  Bayesian  hierarehieal  inferenee  is 
developed  for  them  (Papers  8,  14,  16,  17,  23,  24,  25).  All  papers  but  Paper  23  and  25  are 
published  or  in  press,  and  they  inelude  two  in  Journal  of  Mathematical  Psychology,  two  in 
Psychometrika,  and  one  in  Sociological  Methodology.  All  these  papers  provide  software 
for  conducting  statistical  inference  for  the  models. 

A  third  place  where  Bayesian  software  has  been  provided  is  Paper  (22),  submitted  to 
Journal  of  Statistical  Software.  The  paper  deseribes  "‘CCTpack:  Cultural  Consensus 
Theory  Applications  to  Data.  R  paekage  version  0.99”,  whieh  is  a  GUI  provided  as  an  R 
paekage  that  provides  hierarehieal  Bayesian  inferenee  options  for  a  collection  of  old  and 
newly  developed  CCT  models.  The  paekage  requires  JAGS  (Just  Another  Gibbs  Sampler, 
Plummer,  2012),  which  is  a  freely  available  package  for  using  Markov  Chain  Monte  Carlo 
(MCMC)  methods  to  perform  Bayesian  hierarehieal  inferenee  on  a  wide  class  of  statistical 
models. 


III.  New  CCT  Models 

Almost  all  of  the  work  in  CCT  prior  to  the  last  five  years  concerned  the  General 
Condorcet  Model  (GCM)  for  diehotomous  (True/False)  data.  While  it  is  easy  to  eonduet  a 
True/False  test  with  groups  of  informants,  the  responses  to  questions  in  sueh  a  design 
provide  only  one  bit  of  information.  In  the  period  of  this  grant,  with  some  overlap  with  an 
earlier  grant  from  The  Air  Foree  Office  of  Seientifie  Researeh  to  the  PI,  several  new  CCT 
models  were  invented  along  with  suitable  Bayesian  hierarehieal  inferenee  software.  These 
models  will  be  deseribed  in  the  next  subseetions. 

A.  Models  with  a  Don ’t  Know  Option 

There  are  many  survey  questionnaires  that  provide  questions  with  three  response  options, 
“True,”  “False,”  and  “Don’t  Know”  (DK).  For  example,  seience  questions  on  the  General 
Social  Survey,  for  example  the  2010  General  Soeial  Survey  (in  eollaboration  with  the 
National  Seienee  Foundation)  uses  a  set  of  12  questions,  11  of  whieh  are  True/False 
format  with  a  'Don't  Know'  option  provided.  Evidence  suggests  that  sueh  questionnaires 
can  be  problematie  beeause  a  DK  response  is  often  seored  as  ineorrect,  and  yet  different 
people,  for  example  women  on  average,  have  been  shown  to  be  more  likely  than  others  to 
use  a  DK  response.  In  Paper  (24),  in  press  in  Sociological  Methodology,  the  GCM  was 
modified  to  handle  a  DK  response,  and  the  modification  allowed  different  informants  to 
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have  different  biases  for  using  this  response.  Inferenee  software  was  presented  for  the  new 
model,  and  it  was  applied  to  the  seienee  soeial  survey  questions  mentioned  above  as  well 
as  Palmore’s  (1998)  “Facts  on  Aging  Quiz”.  Covariates  were  also  incorporated  into  the 
model,  and  it  was  shown  that  a  consensus  answer  key  that  corresponded  to  the 
scientifically  correct  answers  was  estimated  despite  isolated  sub-cultures  that  were 
discemable  through  the  covariates. 

In  Paper  (25),  under  review  in  American  Journal  of  Psychology,  the  model  with  a  DK 
response  was  used  on  some  questions  about  “what  constitutes  behavior?”  The  informants 
were  scientists  in  three  societies  of  behavioral  biologists  and  an  additional  group  of 
undergraduate  psychology  majors.  The  questions  posed  acts  of  hypothetical  organisms 
(e.g.  a  heart  pumps  blood,  a  rat  runs  up  a  tree),  and  the  informants  had  to  decide  using  the 
three  response  options  whether  a  particular  act  constituted  behavior.  The  new  CCT  model 
was  able  to  find  a  consensus  definition  of  behavior  as  well  as  to  score  each  behavioral  or 
non-behavioral  act  for  its  cultural  saliency  relative  to  this  definition.  The  paper  proposes 
CCT  more  generally  as  an  approach  to  examining  whether  or  not  a  group  of  scientists 
share  a  consensus  on  the  use  of  a  particular  scientific  term. 

B.  Models  for  Assessing  Ties  in  a  Graph 

In  Paper  (20),  a  model  is  developed  for  assessing  ties  (edges)  in  a  graph,  e.g.  a  social 
network.  In  this  model,  informants  respond  dichotomously  (Present,  Absent)  to  questions 
about  whether  or  not  there  is  a  tie  between  two  nodes  in  a  graph.  One  earlier  CCT  paper 
on  graph  aggregation  was  Batchelder,  Kumbasar,  and  Boyd  (1997).  The  main  addition  to 
this  model  provided  in  Paper  (20)  is  that  the  researcher  can  impose  constraints  on  the 
answer  key  (consensus  graph)  that  are  not  assumed  to  hold  for  the  responses  of  each 
informant.  In  this  paper,  a  two-cluster  balance  structure  (e.g.  Cartwright  and  Harary,  1956) 
is  imposed  on  the  consensus  graph.  One  interesting  facet  of  this  work  is  that  the  addition 
of  a  required  structure  on  the  answer  key  requires  that  a  model-specific  MCMC  sampler 
had  to  be  constructed  to  conduct  the  Bayesian  inference  for  the  model.  Several  studies 
were  run,  for  example  cities  were  selected  out  of  two  states,  and  for  each  pair  of  cities  the 
informant  had  to  say  ‘Same  State’  or  ‘Different  State.’  In  another  study,  informants  were 
given  two  states  and  asked  whether  or  not  they  shared  a  border.  In  this  work,  it  was 
realized  that  tie  judgments  could  be  based  on  either  nodal  knowledge  (e.g.  the  state  where 
the  city  is)  or  tie  knowledge  (is  there  a  shared  border),  or  both.  Paper  (20)  developed  a  de¬ 
based  model;  however,  the  more  difficult  model  to  design  and  estimate  is  a  nodal-based 
model.  The  additional  difficulty  is  due  to  the  fact  that  if  an  informant  knows  the  attribute 
of  a  particular  node,  that  knowledge  is  operative  for  any  tie  in  the  graph  involving  that 
node.  In  the  later  case,  a  nodal  model  has  recently  been  worked  out  along  with  a  model- 
specific  MCMC  sampler,  but  it  has  not  been  tested  against  real  data  or  published  yet. 
Apart  from  the  single  published  paper  mentioned  above,  this  area  is  currently  an  active 
one  in  out  lab. 

C.  Models  for  Ordinal  (Likert  Data) 

One  way  to  collect  more  information  about  a  question  is  to  ask  for  degrees  of  truth  on  an 
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ordered  scale.  One  example  of  ordinal  CCT  data  would  be  for  informants  to  rate  sentences 
of  a  certain  type  on  grammaticality  using  a  seven-point  scale,  from  clearly  grammatical  to 
not  grammatical.  Another  example  is  to  rate  a  particular  city  in  terms  of  its  properties 
(quality  of  weather,  of  nightlife,  etc.).  In  Paper  (16),  in  press  in  Psychometrika  (down 
loadable  from  their  website,  but  not  yet  appearing  in  the  journal),  a  new  CCT  model  for 
ordinal  data  is  provided,  and  applied  to  real  data  from  the  above  two  examples.  The  model 
assumes  that  each  item  occupies  a  latent  place  on  a  continuum,  and  that  each  informant 
draws  an  item  placement  from  a  Gaussian  distribution  centered  on  the  item  placement  and 
with  a  variance  depending  on  the  competence  of  the  informant  and  difficulty  of  the  item. 
Each  informant  chops  the  continuum  into  regions  corresponding  to  the  ordinal  categories. 
Different  informants  can  chop  differently  depending  on  their  biases  for  extreme  or 
middling  ratings  as  well  as  ratings  shifted  up  or  down  the  scale.  In  addition,  the  new 
model  allows  for  one  to  test  for  and  identify  two  or  more  consensus  answer  keys  (latent 
item  locations  on  the  continuum)  from  the  data.  This  addition  to  CCT  models  is  described 
in  some  detail  in  Section  IV  to  follow.  It  was  found  that  in  the  grammaticality  data  there 
was  a  single  consensus  truth;  however,  in  the  cities  data  the  model  was  able  to  estimate  a 
separate  answer  key  for  each  city  tested  (Irvine,  CA;  New  York  City,  Miami). 

The  paper  provides  a  detailed  comparison  between  CCT  modeling  and  IRT  (item  response 
theory)  modeling  popular  in  psychometric  test  theory.  IRT  models  assume  knowledge  of 
the  correct  answers  to  the  questions  and  they  attempt  to  estimate  the  ability  of  the  test 
takers,  on  the  other  hand,  CCT  models  are  interested  in  estimating  the  correct  answers  to 
the  questions  as  seen  by  the  test-takers,  and  only  secondarily  to  estimate  the  ability 
(cultural  competence)  of  the  informants.  Apart  from  these  essential  differences,  both 
theories  use  somewhat  similar  statistical  specifications  of  their  models. 

D.  Models  for  Continuous  Data 

In  Papers  (23)  and  (21),  CCT  models  for  continuous  type  response  data  are  provided. 
Paper  (21)  (in  press  in  Educational  and  Psychological  Measurement)  uses  classical 
Maximum  Likelihood  Estimation  techniques  instead  of  hierarchical  Bayesian  inference, 
and  the  model  is  applied  to  a  published  data  set  consisting  of  experts  rating  50  essays  on  5 
attributes.  Multiple  correspondence  analysis  was  used  to  convert  the  five  ordinal  ratings 
into  a  continuous  scale.  The  model  allows  several  kinds  of  rater  biases  and  item  easiness 
parameters  to  be  estimated.  Fourteen  raters,  including  two  of  the  experts,  provide  the  data 
for  the  model,  and  a  single  consensus  answer  key  was  estimated.  As  expected,  the  model 
is  able  to  pick  up  the  experts  as  the  ones  with  high  competence.  Paper  (19)  provides  some 
available  software  for  these  continuous  models. 

In  Paper  (23)  (under  review  Journal  of  Mathematical  Psychology),  a  hierarchical 
model  for  continuous  data  such  a  probability  judgments  is  constructed.  Its  underpirmings 
are  similar  to  the  ordinal  model  in  Paper  (16)  except  bias  is  represented  differently  in  that 
it  is  applied  to  the  latent  draws  rather  than  to  the  cut  points  of  the  ordinal  model.  Bayesian 
hierarchical  inference  is  developed  for  the  model  (see  Section  II).  The  model  is  applied  to 
forecasting  the  likelihood  of  future  events  and  to  evaluating  various  eating  practices 
connected  with  health  issues.  The  model  was  extended  to  allow  more  than  one  answer  key 
(described  in  Section  IV).  It  turns  out  that  the  model  selected  a  single  answer  key  for  the 
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forecasting  questions  despite  considerable  heterogeneity  in  the  responses;  however,  for  the 
eating-practices-data  there  were  two  discemable  answer  keys,  with  one  indicating 
considerable  concern  and  compliance  with  popular  health  issues. 

IV.  Tests  of  the  One  Culture  Assumption 

Crucial  to  the  successful  use  of  a  CCT  model  is  to  statistically  evaluate  the  evidence  for  a 
single  culture  solution.  If  the  single  culture  assumption  is  rejected,  then  either  the  data 
support  more  than  one  consensus  answer  keys  or  there  is  no  discemable  consensus  in  the 
data.  One  of  the  major  developments  in  the  period  of  the  grant  has  been  the  development 
of  tests  for  one  or  more  than  one  answer  keys.  These  developments  are  seen  in  Papers 
(14),  (16),  (17),  and  (23).  In  the  proposal  for  the  ARO  grant,  the  following  was  written; 

“There  are  two  cases  where  CCT  can  be  used  in  studying  multiple  cultures:  (1)  the  cultural 
membership  of  each  informant  is  obtainable  from  covariates  available  on  the  informants; 
and  (2)  the  cultural  membership  of  each  informant  is  latent  (unobserved).  In  both  cases,  an 
appropriate  CCT  model  would  specify  two  or  more  latent  answer  keys  as  parameters.  The 
second  case  is  the  more  difficult  one  because  the  CCT  model  must  also  specify  latent 
group  membership  parameters.  In  this  case,  the  model  would  take  the  form  of  a  finite 
mixture  model  ...  “ 

In  Paper  (8),  a  major  theorem  for  the  GCM  was  developed  that  bears  directly  on  the  one- 
culture  assumption  of  this  model.  The  key  is  to  construct  a  matrix  of  informant-informant 
correlations  taken  over  items,  say  R  =  (^y)^,^^,  where  r^j  is  the  Pearson  correlation 

between  informants  i  and  j  calculated  over  their  dichotomous  responses  to  the  items  (from 
the  2  by  2  table  consisting  of  the  number  of  items  that  both  said  ‘True,”  i  said  “True”7 
said  “False”,  etc.).  The  theorem  shows  that  the  correlation  matrix  has  a  one-factor 
structure,  that  is  the  off-diagonal  terms  are  approximated  by  ==  a^aj ,  where,  for 

example,  a^  is  the  correlation  between  informant  fs  responses  and  the  latent  consensus 
answer  key.  Such  a  property  is  testable  by  a  suitable  factor  analysis  of  the  correlation 
matrix  followed  by  an  examination  of  the  scree  plot  of  the  eigenvalues  to  see  if  the  data 
supports  a  one-factor  structure.  While  checking  a  scree  plot  for  a  single  factor  structure  is 
generally  handled  with  various  controversial  ‘rules-of-thumb,’  Paper  (14)  developed  a 
graphic  Bayesian  post-predictive  test  (see  paper  for  details)  for  the  one  answer  key 
assumption.  The  test  worked  well  to  discern  when  a  one-culture  assumption  was  met  with 
simulated  data,  and  it  has  been  incorporated  into  the  software  for  the  GCM  model. 

Two  mathematical  developments  of  the  one-factor  test  allowed  tests  to  be  developed  for 
other  models  and  other  numbers  of  cultural  truths.  The  basic  theorem  for  the  case  of 
several  answer  keys  in  the  GCM  states  that  the  correlation  of  two  informants,  i  and  j,  over 
the  items  is  a  product  of  three  terms:  (1)  The  correlation  of  i  with  his  or  her  answer  key 
(like  the  a,  above);  (2)  The  correlation  of  j  with  his  or  her  answer  key;  and  (3)  The 
correlation  between  the  two  answer  keys.  The  third  factor  in  the  above  formulation 
implies  that  the  scree  plot  will  exhibit  more  than  one  factor  in  case  there  is  not  a  single 
cultural  truth,  and  it  was  developed  in  Paper  (14)  into  a  Bayesian  post-predictive  test  for 
multiple  cultural  truths.  This  approach  worked  well  on  simulated  data,  and  it  revealed 
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multiple  cultures  in  several  well-known  anthropological  data  sets  that  were  not  known 
before  the  model  analysis  (see  papers). 

The  results  descrihed  so  far  were  proved  for  the  GCM,  namely  the  main  model  for 
dichotomous  data.  As  a  consequence  it  became  important  to  see  if  this  approach  could  be 
applied  to  the  new  CCT  models  described  Section  III.  It  turned  out  that  this  approach 
either  caries  over  completely  (for  example  for  the  continuous  model)  or  holds  to  a  good 
approximation  for  the  other  models.  So  the  approach  for  detecting  the  number  of  cultural 
truths  based  on  the  scree  plot  of  the  informant-by-informant  correlations  has  been 
incorporated  into  the  Bayesian  hierarchical  software  provided  in  Paper  (22).  Viewed 
generally,  these  developments  have  extended  CCT  to  a  form  of  substantive,  model-based 
clustering,  where  informants  are  clustered  into  consensus  truth  patterns  based  on  their 
responses  to  the  same  set  of  questions. 

V.  Other  Research  Supported  by  the  Grant 

It  turns  out  that  CCT  models  for  categorical  data  (e.g.  the  GCM  and  the  model  for  the 
don’t  know  response)  are  examples  of  Multinomial  Processing  Tree  (MPT)  models,  which 
are  a  major  variety  of  cognitive  models  first  invented  by  the  PI  and  his  students  in  the 
1990s,  e.g.  Batchelder  and  Riefer  (1999).  Several  new  papers  in  this  area  were  published, 
some  with  the  support  of  this  grant.  These  include  Papers  (12),  (13),  and  (18).  Paper  (12) 
is  an  invited  encyclopedia  piece  on  MPT  type  models.  Paper  (13)  develops  Bayesian 
hierarchical  inference  for  one  of  the  most  popular  MPT  models,  and  Paper  (18)  is  a  reply 
in  Psychological  Bulletin  to  some  authors  who  posed  questions  about  the  viability  of  MPT 
models.  In  addition.  Paper  (15)  is  a  major  paper  in  the  area  of  human  problem  solving 
published  in  Journal  of  Problem  Solving  with  some  support  from  the  grant.  While  it  is 
unrelated  to  CCT  models,  it  has  an  interesting  set  of  brainteasers  in  it,  and  it  was  featured 
in  a  keynote  address  by  the  PI  in  a  Dagstahl  Castle  Conference  on  Problem  Solving 
involving  formalists  in  the  computer  sciences  and  psychology. 


VI.  Conclusion 

During  the  period  of  the  grant,  it  has  been  possible  to  greatly  expand  CCT  modeling  well 
beyond  its  usual  application  area  in  cultural  anthropology.  Papers  publishing  CCT  model 
analysis  have  appeared  or  are  under  review  in  anthropology  outlets  (Papers  10,11), 
sociology  outlets  (Papers  20,  24),  psychology  outlets  (Papers  8,  14,  16,  17,  23,  25), 
education  outlets  (Papers  19,  21),  and  statistics  outlets  (Paper  22).  The  Pis  Cognitive 
Psychometrics  Laboratory  has  included  five  undergraduates,  five  graduate  students,  one 
Post  Doctoral  Fellow,  one  Sabbatical  visitor,  and  several  Professors.  Many  of  these 
researchers  gladly  involved  themselves  with  CCT  research  projects,  although  many  of 
them  received  no  financial  support  from  the  ARO  grant.  With  the  new  CCT  Graphic  User 
Interfaces  (Papers  10,11,22),  there  is  hope  that  researchers  who  are  already  using  CCT 
models  will  greatly  improve  their  statistical  inference  potential,  and  newcomers  to  the 
models  will  find  them  useful  for  their  purposes. 
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There  remain  CCT  application  areas  and  model  developments  that  are  in  need  of 
additional  research  and  application.  Many  of  them  fall  into  the  areas  of  aggregating  ties  in 
a  social  or  covert  network,  discerning  events  that  happened  from  eyewitnesses,  and 
detecting  intentional  prevaricators  among  informants.  It  will  be  the  intention  of  the  PI  to 
apply  for  more  funding  from  the  ARO  if  the  results  of  the  current  grant  are  considered  to 
have  a  proper  level  of  importance  for  additional  funding. 
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